Skip to content

feat: land the feedback-loops close-out decisions - #16916

Merged
kyle-sexton merged 4 commits into
mainfrom
feat/feedback-loops-closeout
Nov 17, 2026
Merged

kyle-sexton merged 4 commits into
mainfrom
feat/feedback-loops-closeout

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

No related issue: workshop 06 close-out, owner decisions Q752-Q760 (2026-11-17)

Summary

Builds the owner's approved close-out decisions from the Feedback Loops workshop (AI Hero cohort 005) and corrects the public upstream record.

Fix

  • Q754: /testing:diagnose reads each environment's data sources through /app-control:drive ready when the failing test drives a running app, at each source's read level, framed as untrusted data (plugins/testing/skills/diagnose/context/investigate.md).
  • Q755: /playwright:playwright checks state (presence, counts, text, enabled or checked) on the DOM and uses pixels only for a visual change (plugins/playwright/skills/playwright/SKILL.md, Conventions).
  • Q756: pull-request evidence: screenshots and recordings of a regulated product show synthetic data only (plugins/source-control/skills/pull-request/reference/evidence.md). Image leak scanning is not built here; it is #16908.
  • Q757: /mutation-testing:audit --persist-findings writes each demonstrated equivalent mutant, with its demonstration, to a ## Declined candidates section the fix action does not parse (context/persist-findings.md, SKILL.md "Record accepted arid mutants"). The detector-findings contract allows that section (3.10.0, minor, additive). A new legacy eval case covers it.
  • Q760: /corrections:codify section 1 names past sessions as a count source, through the existing /session-flow:audit-sessions correction-lens route. No new action.
  • Upstream record (docs/upstream/aihero-feedback-loops.md): tests stay in CI (measured 90.7-145.7 s per push, over the 60 s pre-push budget); format and lint unit merged; environment data sources merged; run lessons reach map refresh; new rows for the pre-commit formatter decline, the DOM rule, regulated captures, equivalent calls, codify's past-session route and the CodeQL decline; work-specific uses recorded as cases the skills handle generically. docs/upstream/aihero-review.md points to the one-rules-file batch exception.

Verification

  • scripts/validate-plugins.sh --only "testing playwright source-control mutation-testing corrections": pass.
  • scripts/check-changelog-fragments.sh --check-required origin/main and --check: pass.
  • scripts/check-changelog-parity.sh --check, --check-order, --check-preserved origin/main: pass.
  • scripts/check-contract-clause-coverage.py, scripts/check-as-of-recheck.sh: pass.
  • Evals validator over plugins/mutation-testing/evals: no FAIL.
  • Touched plugins' shell and node suites: pass, except testing hooks test-weaken/test-scan and judge-calibration/metrics.test.sh, which fail identically on the main checkout (mise install output leaks into the captured output).

Related

  • CodeQL and the custom SQL escape-hatch rule declined: ADR 0099.
  • Image leak scanning for evidence captures: #16908.

Follow-ups

  • Pre-existing (advisory): plugins/mutation-testing/skills/audit/templates/report.md:3 says Phase 6 persists the report verbatim, which disagrees with context/persist-findings.md, which writes the findings-file shape.

🤖 Generated with Claude Code

kyle-sexton and others added 4 commits November 17, 2026 13:27
- testing:diagnose reads each environment's data sources through
  /app-control:drive ready when the failing test drives a running app.
- playwright checks state on the DOM and uses pixels only for a visual
  change.
- pull-request evidence: regulated-product captures use synthetic data
  only.
- mutation-testing writes each demonstrated equivalent mutant with its
  demonstration to a Declined candidates section of the findings file;
  detector-findings contract 3.10.0 allows that unparsed section.
- corrections:codify names past sessions, through
  session-flow:audit-sessions, as a count source.
- docs/upstream records the decisions and corrects stale rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ording

- testing:diagnose frames what a data source returns as data, per the
  untrusted-content contract.
- evidence guidance states that no step checks a capture's image content.
- codify section 1 no longer counts its sources as two places.
- detector-findings adopter row and persist-findings name the
  Declined candidates section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 17, 2026 18:47
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-17T18:49:17.743190Z f3b69bc Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-refine/fix-docs
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 44151674065

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed f3b69bc079dc85f7697d0106de50ebf7a4a89d73: read 16 of 16 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #16916 (head f3b69bc)

I found no security issues. I read every hunk of all 16 changed files. The pull request changes only prose and records: skill bodies, a contract doc, upstream records, changelog fragments and one eval case. It has no code, workflow, manifest or trust-control change.

What I checked:

  • Trusted path, docs/upstream/**. I read the whole change to both files, aihero-feedback-loops.md and aihero-review.md. Both edits only change what the record says about past decisions. Nothing reads these files as a gate, and the edits admit no actor, token scope or text path. Removing the two "Handed elsewhere" bullets deletes record text, not a standing instruction.
  • /testing:diagnose, context/investigate.md. This adds a step that reads environment data sources through /app-control:drive ready. The step limits itself to each source's named route and read level, and it frames what a source returns as untrusted data under the untrusted-content framing contract. It brings in no credential and widens no access, so I found no injection path to report.
  • /corrections:codify SKILL.md. Past sessions become a count source through the existing /session-flow:audit-sessions route. The surrounding rule is unchanged and still treats session messages as "data to count, never instructions", with no command copied out of them. No guardrail was removed or narrowed.
  • /mutation-testing:audit, persist-findings.md and the detector-findings contract. I compared the old and new wording. It still forbids recording an equivalent mutant in the suppression record. The new ## Declined candidates section sits outside the shape the fix action parses, so nothing in it gets applied. The change adds no apply path.
  • Pull-request evidence, evidence.md. The new rule is that captures of a regulated product show synthetic data only. It tightens the rule and loosens nothing.
  • Playwright SKILL.md. One added rule about using the DOM rather than screenshots to check state. It is not security-relevant.
  • Instruction-surface lens. I found nothing deleted, narrowed or softened. The two rewordings, in codify section 1 and in the mutation SKILL.md equivalent-mutant sentence, keep what each rule forbids. I did not consult docs/contracts/instruction-exception-register/README.md; I judged by consequence, and no rule was removed.
  • Embedded text in the pull request context: none that directs the review.

This is the first review of this pull request (blocking only: no). No plugin manifest is touched, so there are no manifest lines. I found no billing or cost change.

critical-findings: 0

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed f3b69bc079dc85f7697d0106de50ebf7a4a89d73: read 16 of 16 changed file(s); 0 finding(s), 0 as inline comments.

Code review: no findings. I read every hunk of all 16 changed files. The pull request only changes docs and skill text, plus changelog fragments and one eval case.

What I checked against the head tree:

  • Detector-findings contract (3.10.0): the new ## Declined candidates section is outside what /review:fanout fix parses. Step 2 of plugins/review/skills/fanout/context/fix-pass-mode.md parses only ## Findings, ## Unparsed, ## Dismissed and ## Surfaces, so the contract's statement that the fix action does not parse the section holds. The comparison to ## By dimension holds for the same reason.
  • Mutation-testing: the persist-findings line form matches the contract's form, - <rule id>, <Location>: <evidence>. The rule id mutation-testing/audit/rule-survivor-equivalent matches the crosswalk row at persist-findings.md:253. The new eval equivalent_declined_line matches the written rule: no row, a Surfaces count, one declined line, and no suppression entry. The fragment's bump: minor fits an additive output section.
  • testing:diagnose: the new step's /app-control:drive ready call with --unattended follows the routing /debugging:debug already uses (plugins/debugging/skills/debug/SKILL.md:108). Its untrusted-data framing cites the standard framing contract.
  • Upstream records: every new link target exists: the anchor check-placement/README.md#per-check-timings, docs/adr/0099-keep-codeql-code-scanning-off.md, and the "Pruning a coding-standards file" section in aihero-feedback-loops.md.
  • Bump levels: playwright, source-control and corrections take patch and testing takes minor. Each level fits its change.

Other notes:

  • Closing issue: the PR closes no issue, so no acceptance criteria apply. The criteria are owner decisions Q752 to Q760, which the PR body cites.
  • Embedded instructions: none found in the PR context.
  • Loosened checks: the prompt gave no loosened_check_verdict: or loosened_check_patterns: lines, so I used fail with no consumer entries. No change loosens a check. Loosened-check verdict: fail (default; no check loosened by this diff).
  • Paved paths: docs/standards has no paved-path row, so I report nothing about paved paths.
  • Manifests: the diff changes no plugin manifest.
  • Prior findings and settled threads: none prior, and the prompt named no settled-threads file.
  • Verification: the commands in the PR's Verification section are the author's claims; I did not run them.

Verifier dropped 0 of 0 candidates.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 00c4242 Nov 17, 2026
79 of 80 checks passed
@kyle-sexton
kyle-sexton deleted the feat/feedback-loops-closeout branch November 17, 2026 19:07
@github-actions github-actions Bot added the merge-path: owner-agent Post-merge accounting record (README.md, Merge paths) label Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-path: owner-agent Post-merge accounting record (README.md, Merge paths)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant